Identifying and prioritizing do-not-do recommendations in Dutch primary care

Background Low-value care provides minimal or no benefit for the patient, wastes resources, and can cause harm. Explicit do-not-do recommendations in clinical guidelines are a first step in reducing low-value care. The aim of this study was to identify and prioritize do-not-do recommendations in general practice guidelines with priority for implementation. Methods We used a mixed method design in Dutch primary care. First, we identified do-not-do recommendations through a systematic assessment of 92 Dutch guidelines for general practitioners (GPs), resulting in 385 do-not-do recommendations. Second, we selected 146 recommendations addressing high prevalent conditions. Third, a random sample of 5000 Dutch GPs was invited for an online survey to prioritize recommendations based on the prevalence of the condition and low-value care practice, potential harm, and potential cost reduction on a scale from 1 to 5/6. Total scores could range from 4 to 22. Recommendations with a median score > 12 were included. In total, 440 GPs completed the survey. Results The selection process led to 30 prioritised recommendations. These covered drug treatments (n = 12), diagnostics (n = 10), referral to other healthcare professions (n = 5), and non-drug treatment (n = 3). Conclusion Dutch clinical guidelines include many do-not-do recommendations that are perceived as highly relevant by the GPs. The list of 30 high-priority do-not-do recommendations can be used to raise awareness of low-value care among GPs. As the recommendations are supported with the latest evidence from international studies, primary healthcare professionals and policy makers worldwide can use the list for further validating the list in their local context and designing strategies to reduce low-value care.


Background
In all sectors of medical practice healthcare professionals provide low-value care procedures and treatments [1]. Low-value care can be defined as care that provides minimal or no benefit, considering harms, costs, alternatives, and patient preferences [2]. De-implementation of lowvalue care can improve the quality of care and reduce costs [3]. Various initiatives have been started to identify low-value care practices. Examples are the 'Choosing Wisely' campaign in various countries, 'Smarter Medicine' in Switzerland, and a list developed by the United Kingdom's National Institute for Health and Care Excellence (NICE) [4][5][6]. Multiple lists with do-not-do recommendations have been compiled [7][8][9][10]. These recommendations advise clinicians to refrain from diagnostics and Open Access BMC Primary Care *Correspondence: Simone.vandulmen@radboudumc.nl treatments that are proven of low value. These lists are mainly focused on hospital care and include only a few numbers of do-not-do recommendations for primary care such as in Switzerland, Canada and the United States [8,11,12].
In the Netherlands, medical specialists [10] and nurses [13] have created lists containing respectively 1366 and 66 low-value care practices. With these lists the awareness of low-value care practice can be increased as a first step in de-implementation [10,14]. Dutch GPs have an important role as gatekeeper to hospital-and specialist care. They are the first healthcare providers for patients with a wide range of conditions. All Dutch citizens are registered with a GP and 78% consult their GP in a year time [15]. GPs can therefore have a significant impact in reducing low-value care to prevent needless harm to patients and to reduce healthcare costs. Well-known examples of low-value care practices in primary care are imaging in patients with non-specific low back pain when red flags are not present, [16] and prescribing antibiotics in patients with non-severe pharyngotonsillitis [17]. Dutch GPs revealed that 99% of them are aware that lowvalue care is provided in primary care and two-thirds (67%) responded that it occurs regularly or often [18]. Unnecessary medication (27%), laboratory tests (25%), such as vitamin and PSA tests, and imaging (18%) were most frequently mentioned low-value care practices [18].
However, it is still challenging for GPs to refrain from providing low-value care. Increased cost-consciousness and awareness of low-value care among GP's is associated with providing less low-value practices [19]. Therefore, a list of do-not-do recommendations in general practice can guide de-implementation activities. The aim of this study is to select do-not-do recommendations with high priority for de-implementation based on the opinion of GPs on prevalence, harm to patients, and potential cost reduction.

Design and setting
The Dutch College of General Practitioners has a longstanding guideline program since 1989. In 2018, 134 guidelines were available. We used the complete set of guidelines to identify recommendations stating that specific interventions should be avoided. We aimed to select and to prioritise do-not-do recommendations in a stepwise approach in three steps [7].
Step 1: Identifying do-not-do recommendations We started with all the 134 guidelines that were available in the database of the Dutch College of General Practitioners in September 2018 and endorsed by the Dutch College of GPs. Fifteen guidelines were published before 2008 and 27 guidelines were being revised or planned for revision in 2018 or in 2019. These guidelines were excluded to prevent the inclusion of recommendations that are not applicable to current practice. In total 92 guidelines were screened for do-not-do recommendations.
A do-not-do recommendation was defined as a recommendation that instructs the GP not to offer specific interventions. We first randomly selected 5 practice guidelines that were analysed by three researchers (JM, LW and EV) to identify do-not-do recommendations. The researchers compared their results and consulted three other researchers (SAvD, RBK and TW) until agreement was reached. Subsequently, we repeated this process for 5 randomly selected guidelines to further refine the inclusion and the exclusion criteria. To evaluate the agreement, another ten guidelines were independently screened to determine the inter-rater reliability. Inter-rater reliability was analysed by calculating Fleiss' Kappa (k) for multiple raters [20]. Using the method described, the remaining 72 guidelines were screened by one researcher (JM). Difficulties in interpretation were discussed with a second researcher (SAvD or RBK) and when necessary, with a third researcher (TW) until consensus was reached.
We included recommendations that applied to practice usually performed by GPs. Elective low-value procedures, such as fundoscopy for diagnosing eye problems, were excluded, as they are only performed by few (specialised) GPs in the Netherlands. Furthermore, recommendations containing only an advice for the patient or stating the obvious or a well-known contra-indication or interaction, were excluded after discussion with the second or third researcher. For example, 'Stop using nifedipine for Raynaud's syndrome if the treatment is ineffective and/ or causes undesirable side effects' (Guideline Raynaud's syndrome, 2018). Recommendations just stating wellknown contra-indications or interactions were excluded. For example, 'Do not prescribe NSAID (or acetylsalicylic acid) to patients who had an anaphylactic reaction to NSAID in the past' (Guideline Pain, 2018). We combined similar recommendations found in one guideline and removed duplicates found in multiple guidelines.
Step 2: Selecting do-not-do recommendations In the second step, we reduced the number of recommendations in order to have an appropriate number feasible for step three. Recommendations were divided among two researchers (NHT and TW, who is a practicing GP and in charge of the College of General Practioners) and screened independently. Recommendations regarding low-prevalent diagnostics, treatments, procedures and referrals were excluded. The prevalence was estimated based on NHT and TW's experienced frequency of recommendation-related visits to the GP practice. Symptoms or illnesses that GPs encounter approximately less than once a month were excluded. Any uncertainties were discussed with a second and third researcher (RBK and SAvD) until consensus was reached. We categorised the recommendations in diagnostics, drug treatments, non-drug treatments, referrals, and miscellaneous.
Step 3: Prioritizing do-not-do recommendations by GPs We used an online survey to assess the opinions of GPs in regard to four criteria of the recommendation based on previous research. [7,21]: 1) prevalence of the condition, 2) prevalence of the low-value care practice, 3) potential harm to the patient, and 4) potential for cost reduction. The online survey was pilot tested by a team of eight researchers, GPs and medical students). After feedback and refinements, the do-not-do recommendations selected in step 2 were randomly distributed into five different online surveys designed in LimeSurvey Version 2.06+ to keep the length of the survey limited and thereby increase the response rate. Invitations for each online survey were sent by the office of the Dutch College of General Practitioners to 1000 GPs, using random sampling of the national database of their members. The majority of the Dutch GPs are member of the Dutch College of General Practitioners. That database consists of 12.766 GPs. Doctors in training for GP were excluded. We aimed for a response of a minimum of 100 GPs for each recommendation to be able to estimate the support for de-implementation of the low-value care practice. Considering the estimated duration of the online survey (20 minutes) the response rate was estimated at 10% based on previous experiences. We developed five surveys (in Dutch) and randomly assigned the GPs to one of the five surveys (1000 per survey). In total 5000 GPs were invited to evaluate the do-no-do recommendations. Table 1 describes the questions and answering options used in the survey. We used a scale from 1 (never) to 6 (very often) for assessing the prevalence of the clinical condition (criterium 1) and the estimation of the prevalence of providing the mentioned low-value care practice by GPs (criterium 2) on a scale from 1 (never) to 6 (very often). The potential harm to the patient (criterium 3) was assessed on a scale from 1 (none) to 5 (major). The potential cost reduction (criterium 4) was assessed on a scale from 1 (don't agree) to 5 (fully agree), and with an 'I don't know' option. The latter answering option was excluded from the analyses. Non responders received one reminder to complete the online survey. The mean score with a standard deviation (SD) and the median with an interquartile range (IQR) were calculated for each criterium. Total scores could range from 4 to 22. We aimed to identify high priority recommendations, and a manageable number of recommendations to communicate to GPs and where GPs can choose from for potential future de-implementation strategies. Therefore, recommendations with a median score of 13 or higher were selected. The results were analysed with SPSS Statistics 25. Figure 1 shows the flow diagram of the identified guidelines and recommendations. In step 1, 92 guidelines were assessed for do-not-do recommendations. Seven guidelines did not include any do-not-do-recommendation. The calculated Fleiss Kappa for assessing inter-rater reliability was 0.715, indicating acceptable agreement [22]. In step 2, from the 385 do-not-do recommendations identified, 3 duplicate recommendations were removed and 236 recommendations were excluded based on low prevalence. This resulted in in 146 do-not-do recommendations included in the online survey. These recommendations were extracted from 53 guidelines with a range of 1 to 8 do-not-do-recommendations per guideline. The majority of the 146 recommendations (88%) concerned care that should not be provided at all, whereas others (12%) referred to care that could be offered with restraint. Of all recommendations, 49% covered drug treatment, 33% diagnostics, 8% non-drug treatment, 4% miscellaneous (e.g. follow-up appointments, combinations of diagnostics and drug treatment), 4% referral to specialist care and 2% covered referral to an another healthcare provider in primary care, such as a physiotherapist. We did not find a clear pattern between the year of publication and the number of recommendations. Guidelines with many do-not-do recommendations in step 2 were the guidelines 'Lumbar radiculair syndrome' (n=8), ' Acute rhinosinusitis' (n=7), 'Pain' (n=7), ' Acute otitis media' (n=7), and Thyroid disorders' (n=7).

Results
In total 440 GPs completed one out of the five online surveys. Four surveys contained 29 do-not-do recommendations and one survey contained 30 recommendations. Each recommendation was assessed by an average of 88 GPs (response rate 8.8%), with a range of 67-108 GPs. The mean age of the GPs was 46.7 years (SD 10.1; range 28-71 years), which is representative for the Dutch population of GPs [23]. The number of respondents was representative with regard to the distribution of GPs per province [23]. Female GPs were over-represented in our study; 67% in our study compared to 58% of all Dutch GPs [24]. The mean percentages female GPs ranged from 63% -75% between the five survey groups.
All do-not-do recommendations with a median score of 13 or higher were selected, resulting in a total of 30 recommendations from 18 guidelines (see Table 2). Five recommendations concerned low back pain from the guidelines 'Lumbar radicular syndrome' and 'Non-specific low back pain' . Both guidelines recommend against imaging and prescription of benzodiazepines. Four recommendations were included from the guideline 'Nontraumatic knee complaints' and three recommendations were included from the guidelines 'Red eye and eye trauma' as well as ' Acute otitis media' . The 30 recommendations covered 33% diagnostics (n=10) and 66% treatment interventions (n=20), including drug treatments      (n=12), referral to other healthcare professions (n=5) and non-drug treatment (n=3).
Concerning the prevalence of the low-value care practice, high scoring recommendations were refill prescription of opiates (60% of the GPs prescribed them frequently to very often), and antibiotics for an infectious conjunctivitis due to a banal pathogen (51% of the GPs prescribed them frequently to very often). Concerning the prevalence of symptoms/clinical picture, the recommendations with the highest median score of 5 were imaging in patients with non-specific low-back pain, oral antibiotics in children with acute otitis media, and decongestive nasal spray as treatment for acute otitis media in children. Ten recommendations had the highest median score of 5 on the potential for cost reduction (criterium 4), with the highest scored recommendation imaging of low-back pain (89% indicated somewhat/fully agree on the potential for cost reduction).

Discussion
We identified 385 do-not-do recommendations in 92 Dutch general practice guidelines. Considering the prevalence, potential harm to patients, and cost reduction, 30 do-not-do recommendations with highest priority for deimplementation activities were selected. This list can be used to increase awareness of low-value care among GPs and for de-implementation strategies.
Some of our do-not-do recommendations were also included in the lists of other countries as part of the Choosing Wisely campaign or comparable initiatives. The most mentioned recommendation is not ordering imaging in patients with non-specific low back pain without red flags [12,[25][26][27]. Another common recommendation is not prescribing benzodiazepines for several indications [25,28,29]. In our list the recommendation against the use of benzodiazepines was selected for lowback pain problems. Four recommendations on our list concern antibiotics prescription. Although antimicrobial resistance and antimicrobial use in the Netherlands are among the lowest in the world, [30,31] GPs noticed that unnecessary prescription of antibiotics remains an important topic.
In a recent study of Kool et al. in the Netherlands, showed a wide practice variation of ordering an X-ray [32]. In order to reduce this low-value care service, A survey on primary care clinicians' perspectives on reducing low-value care showed that GPs need more time for a good explanation to the patient, and education for both the GPs and other healthcare providers as well as patients [18]. These findings confirm the topics that have been identified in our study and that further de-implementation activities are needed. Successful examples of de-implementation strategies for reducing imaging for patients with low-back pain exist, and they focus on both clinicians and patients [33,34].

Strengths and limitations
A strength of this study was the systematic identification of recommendations from all recent national guidelines, specifically for GPs. Therefore, the list covers a broad scope of conditions seen in GP care practice. We asked GPs to score the relevance for GP practice so the list is also a good reflection of common practice based on the guideline recommendation. This is a crucial condition for actual de-implementation, as the guidelines are integrated in national quality policy and implemented using multiple strategies such as in CME, audit and feedback, patient education, and practice accreditation [35]. There are several limitations of the methodology of our study. Firstly, we did not assess the evidence for each recommendation, nor did we quantify the prevalence and actual cost. Secondly, all guidelines were screened based on the exact formulation in guideline, which might be arbitrary in some cases. For example: "Strive to prescribe opioids for the shortest duration of time. " is a positive formulated recommendation, whereas "Do not prescribe opiates longer than needed. " is formulated as a do-not-do recommendation. We therefore might have missed lowvalue care practices that were not specifically formulated as a do-not-do recommendations. Finally, for prioritizing we had an average response rate of 8.8% and therefore the results may be influenced by non-response bias. On the other hand, the absolute number of respondents is acceptable and similar as in other surveys among GPs conducted on a national level. Moreover, the respondents were representative concerning age and location of the practices across the Netherlands.

Implications for research and practice
Changing behavior in clinical practice is difficult due to several barriers on the level of the health care provider, the patient, social context, and the organizational and political context [36]. Patients often expect diagnostic certainty and hope for treatment interventions. This may be due to fear of serious illness and lack of knowledge [37,38]. Other barriers to reducing low-value care practices are GPs time constraints, community standards, lack of tools and communications skills to support shared decision making, and fear for being sued [11,18,19,37]. Increased cost-consciousness and awareness of low-value care among GPs is associated with providing less low-value practices [19]. The development of a list of low-value care services is the first step in reducing these services in clinical practice. The de-implementation strategies should be tailored to the identified local barriers and facilitators as for each recommendation the influencing factors are different in the local context. The selected do-not-do recommendations can be used by GPs and policy makers worldwide, as they are supported with the latest evidence from international studies. Although these recommendations are probably not formulated in each international guideline, the list can be used to create awareness on low-value care and for designing strategies to reduce low-value care in other countries as well. Future qualitative research with GPs on their thoughts about this list could contribute to further validity of the list. Future research could focus on the volume and variation of these low-value care practices, the barriers to reducing the low-value care, and what GPs need to change their practice. Routine monitoring with administrative data with sufficient clinical detail to assess appropriateness of care and risk adjustment is necessary to estimate the magnitude of the problem and the potential cost reduction. The developed list with do-not-do recommendations with a high priority for de-implementation activities according to the GPs could be used as a starting point for measurement of low-value care practices, and to raise awareness on low-value care amongst GPs [35,39]. This routine monitoring should primarily facilitate GPs to learn about their own practice and stimulate them to improve. In addition, concrete plans on how to de-implement this low-value care are needed to change practice. For a structural change systematic, repeated and purposeful efforts are required [40]. For further implementation research we will collaborate with the Dutch College of General Practitioners to start de-implementation projects based on this list, starting with the assessment of some of these low-value care practices. It is important to disseminate the list with a targeted communication campaign in order to raise awareness amongst GPs and patients. Engaging patient organizations and patients in de-implementation activities, e.g. through educational materials or tools for shared decision making, could help to reduce low-value care [41].

Conclusion
Dutch clinical guidelines include many do-not-do recommendations that are perceived as highly relevant by the GPs. The list of 30 do-not-do recommendations identified by GPs can be used to raise awareness of low-value care among all GPs. As the recommendations are supported with the latest evidence from international studies, primary healthcare professionals and policy makers worldwide can use the list for further validating the list in their local context and designing strategies to reduce low-value care.